System and method for compressing and decompressing images using block-based compression format

ABSTRACT

Disclosed herein includes a system, a method, and a device for compressing image data. The device includes one or more processors, coupled to memory, configured to identify a plurality of sub-blocks of a block of image data including a first sub-block and a second sub-block. The one or more processors are configured to identify a first data characteristic of data of the first sub-block and a second data characteristic of data of the second sub-block, determine a first compression technique based at least on the first data characteristic of the first sub-block, determine a second compression technique based at least on the second data characteristic of the second sub-block, and compress the first sub-block using the first compression technique and the second sub-block using the second compression technique.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of priority under 35 U.S.C. § 120 asa continuation of U.S. Non-Provisional patent application Ser. No.16/820,264, filed on Mar. 16, 2020, the content of which is incorporatedherein by reference in its entirety.

FIELD OF DISCLOSURE

The present disclosure is generally related to a system and method forcompressing and decompressing image data, including but not limited to asystem and method for compressing different sub-blocks of a block ofimage data by optimally selecting between different possible compressiontechniques based on data characteristics of the sub-blocks.

BACKGROUND

One challenge in image data compression relates to reducing storagerequirements in performing image data compression. For example, asignificant storage space may be required to store pixel surface datarepresenting a texture space of image data. Here, the term “surfacedata” refers to data relating to a 3D display window texture (e.g., are-projectable 3D display window texture). Another challenge in imagedata compression relates to reducing power consumption in transmittingimage data from an image processing system to another system. Forexample, significant power may be required to transmit texel surfacedata from a host processor to a display module via a high-speed serialinterface. In view of the needs for efficient storage utilization andpower consumption, improvements in compression algorithm and data formatfor image data may be desired.

SUMMARY

Various embodiments disclosed herein are related to method forcompressing image data. In some implementations, the method includesidentifying, by a processor, a plurality of sub-blocks of a block ofimage data including a first sub-block and a second sub-block. Themethod may further include identifying, by the processor, a first datacharacteristic of data of the first sub-block and a second datacharacteristic of data of the second sub-block. The method may furtherinclude determining, by the processor, a first compression techniquebased at least on the first data characteristic of the first sub-block.The method may further include determining, by the processor, a secondcompression technique based at least on the second data characteristicof the second sub-block. The method may further include compressing, bythe processor, the first sub-block using the first compression techniqueand the second sub-block using the second compression technique.

In some implementations, the second compression technique determinedbased on the second data characteristic that is different from the firstdata characteristic, may be different from the first compressiontechnique.

In some implementations, the data characteristic of each of the firstand second sub-blocks may be at least one of an image quality, a rangeof values, a compression rate, or a distortion efficiency. The methodmay further include determining a range of values within each of thefirst and second sub-blocks such that the range of values satisfy atarget cost function of (1) a compression rate and (2) a distortionefficiency.

In some implementations, each of the first and second compressiontechniques may be one of lossless compression, lossy compression,prediction-based compression, or no compression.

In some implementations, the block of image data may correspond to oneof a plurality of channels. Each channel may be one of a color channelor an alpha channel.

In some implementations, the method may further include generating aplurality of first compressed blocks by compressing a plurality of firstblocks. Each block of the plurality of first blocks may correspond toone of the plurality of channels. The method may further includedividing the plurality of first compressed blocks into a plurality offirst data slices. Each of the plurality of first data slices mayinclude a plurality of portions, such that each portion includescompressed blocks corresponding to a respective one of the plurality ofchannels. In some implementations, the method may further includegenerating a plurality of second compressed blocks by compressing aplurality of second blocks. Each block of the plurality of second blocksmay correspond to one of the plurality of channels. The method mayfurther include dividing the plurality of second compressed blocks intoa plurality of second data slices. Each of the plurality of second dataslices may include a plurality of portions, such that each portionincludes compressed blocks corresponding to a respective one of theplurality of channels. The plurality of first blocks may have a firstMIP level different from a second MIP level of the plurality of secondblocks. Here, the term “MIP” refers to a multi-resolution scale factorrepresentation commonly used in computer graphics, standing for “Multumin Parvo”.

In some implementations, the method may further include generating,based on the plurality of first data slices, an indirection tableindicating address information of compressed blocks in the plurality offirst data slices.

Various embodiments disclosed herein are related to a device forcompressing image data may include one or more processors, coupled tomemory. In some implementations, the one or more processors may beconfigured to identify a plurality of sub-blocks of a block of imagedata including a first sub-block and a second sub-block. The one or moreprocessors may be further configured to identify a first datacharacteristic of data of the first sub-block and a second datacharacteristic of data of the second sub-block. The one or moreprocessors may be further configured to determine a first compressiontechnique based at least on the first data characteristic of the firstsub-block. The one or more processors may be further configured todetermine a second compression technique based at least on the seconddata characteristic of the second sub-block. The one or more processorsmay be further configured to compress the first sub-block using thefirst compression technique and the second sub-block using the secondcompression technique.

In some implementations, the second compression technique determinedbased on the second data characteristic that is different from the firstdata characteristic, may be different from the first compressiontechnique.

In some implementations, the data characteristic of each of the firstand second sub-blocks may be at least one of an image quality, a rangeof values, a compression rate, or a distortion efficiency. The processormay be further configured to determine a range of values within each ofthe first and second sub-blocks such that the range of values satisfy atarget cost function of (1) a compression rate and (2) a distortionefficiency.

In some implementations, each of the first and second compressiontechniques may be one of lossless compression, lossy compression,prediction-based compression, or no compression.

In some implementations, the block of image data may correspond to oneof a plurality of channels. Each channel may be one of a color channelor an alpha channel.

In some implementations, the one or more processors may be furtherconfigured to generate a plurality of first compressed blocks bycompressing a plurality of first blocks. Each block of the plurality offirst blocks may correspond to one of the plurality of channels. The oneor more processors may be further configured to divide the plurality offirst compressed blocks into a plurality of first data slices. Each ofthe plurality of first data slices may include a plurality of portions,such that each portion includes compressed blocks corresponding to arespective one of the plurality of channels. In some implementations,the one or more processors may be further configured to generate aplurality of second compressed blocks by compressing a plurality ofsecond blocks. Each block of the plurality of second blocks maycorrespond to one of the plurality of channels. The one or moreprocessors may be further configured to divide the plurality of secondcompressed blocks into a plurality of second data slices. Each of theplurality of second data slices may include a plurality of portions,such that each portion includes compressed blocks corresponding to arespective one of the plurality of channels. The plurality of firstblocks may have a first MIP level different from a second MIP level ofthe plurality of second blocks.

In some implementations, the one or more processors may be furtherconfigured to generate, based on the plurality of first data slices, anindirection table indicating address information of compressed blocks inthe plurality of first data slices.

Various embodiments disclosed herein are related to a non-transitorycomputer readable medium storing program instructions for causing one ormore processors to identify a plurality of sub-blocks of a block ofimage data including a first sub-block and a second sub-block. In someimplementations, the one or more processors may be further caused toidentify a first data characteristic of the first sub-block and a seconddata characteristic of data of the second sub-block. The one or moreprocessors may be further caused to determine a first compressiontechnique based at least on the first data characteristic of the firstsub-block. The one or more processors may be further caused to determinea second compression technique based at least on the second datacharacteristic of the second sub-block. The one or more processors maybe further caused to compress the first sub-block using the firstcompression technique and the second sub-block using the secondcompression technique.

In some implementations, the data characteristic of each of the firstand second sub-blocks may be at least one of an image quality, a rangeof values, a compression rate, or a distortion efficiency.

These and other aspects and implementations are discussed in detailbelow. The foregoing information and the following detailed descriptioninclude illustrative examples of various aspects and implementations,and provide an overview or framework for understanding the nature andcharacter of the claimed aspects and implementations. The drawingsprovide illustration and a further understanding of the various aspectsand implementations, and are incorporated in and constitute a part ofthis specification.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are not intended to be drawn to scale. Likereference numbers and designations in the various drawings indicate likeelements. For purposes of clarity, not every component can be labeled inevery drawing.

FIG. 1 is a block diagram of an embodiment of a system for processingimage data, according to an example implementation of the presentdisclosure.

FIG. 2 is a block diagram of an embodiment of a computing system,according to an example implementation of the present disclosure.

FIG. 3 is a representation of a block of image data for performing datacompression, according to an example implementation of the presentdisclosure.

FIG. 4 is a table indicating parameters for performing data compression,according to an example implementation of the present disclosure.

FIG. 5 is a table indicating a format of a block header as a result ofperforming data compression, according to an example implementation ofthe present disclosure.

FIG. 6 is a representation of image data as a result of performing datacompression, according to an example implementation of the presentdisclosure.

FIG. 7 is a block diagram of an embodiment of an encoder for compressingimage data, according to an example implementation of the presentdisclosure.

FIG. 8 is a block diagram of another embodiment of an encoder forcompressing image data, according to an example implementation of thepresent disclosure.

FIG. 9 is a block diagram of an embodiment of a system for processingimage data, according to an example implementation of the presentdisclosure.

FIG. 10 is an indirection table for performing decoding of image data,according to an example implementation of the present disclosure.

FIG. 11 is a flow chart illustrating a process to compress a block ofimage data, according to an example implementation of the presentdisclosure.

DETAILED DESCRIPTION

It should be understood that the terminology used in the presentdisclosure is for the purpose of description only and should not beregarded as limiting.

Disclosed herein include embodiments of a system and method forcompressing image data, including but not limited to a system and methodfor compressing different sub-blocks of a block of image data byoptimally selecting between different possible compression techniquesbased on data characteristics of data of the sub-blocks.

One challenge in image data compression relates to reducing storagerequirements in performing image data compression. For example, asignificant storage space may be required to store texel surface datarepresenting a texture space of image data. Another challenge in imagedata compression relates to reducing power consumption in transmittingimage data (e.g., processed image data) from one image processing systemto another system. For example, significant power may be required totransmit texel surface data from a host processor to a display modulevia a high-speed serial interface. In view of the needs for efficientstorage utilization and power consumption, improvements in compressionalgorithm and data format for image data are desired. In one aspect,compression of a large size of image data can be performed using asingle compression technique while the data may contain color channelsthat have different image characteristics (or data characteristics) interms of compression efficiency.

To solve this problem, in some implementations, instead of encodingdifferent color channels together using the same encoding method, animage processing system may utilize a block-based image compressionformat thereby to fetch color channels independently from image data(e.g., texture data) and encode the color channels independently. Insome implementations, an image processing system may compress image datacontaining blocks or sub-blocks based on different image characteristics(or data characteristics) thereof to achieve high energy efficiency andhigh memory utilization. In some implementations, a block of image datacan be divided into multiple sub-blocks and each sub-block can becompressed independently. For example, different sub-blocks havingdifferent image characteristics (or data characteristics) in terms ofcompression efficiency can be compressed in different compression modes,e.g., by optimally selecting between different possible compressiontechniques. A compression mode of each sub-block can be determined basedon image characteristics (or data characteristics) of that sub-block,such as image quality, a range of values, compression rate, distortionefficiency, and so on. In some implementations, each channel of imagedata (e.g., color channels or alpha channel) can be compressedindependently so that different channels of the same image block can becompressed in different compression modes, e.g., by optimally selectingbetween different possible compression techniques. A compression mode ofeach channel in the same image block can be determined based on imagecharacteristics (or data characteristics) of that channel, such as imagequality, a range of values, compression rate, distortion efficiency, andso on.

In some implementations, an image processing system may perform aplan-of-record (or palette-based) quantization compression algorithmwith an indirection table for quick random access to any channel's blockof texture data. In some implementations, an indirection tableindicating address information of compressed blocks in a plurality ofdata slices, may be generated to be used by a decoder which needs toaccess the blocks not in the original raster scan order. In someimplementations, sizes of blocks may be embedded with the blocks in ashort header. In some implementations, an image processing system mayidentify or extract sizes of blocks embedded in a short header and thenuse them to generate an indirection table on-the-fly. In someimplementations, an image processing system may have an encoder that isconfigured to generate and compress a plurality of MIP levels in texturedata, for example, up to 4 MIP levels. Here, the term “MIP” refers to amulti-resolution scale factor representation commonly used in computergraphics, standing for “Multum in Parvo”. In some implementations, ablock of image data may be arranged as a plurality of sub-blocks. Forexample, a block of 16×16 pixels may be arranged as 2×2 sequence ofsub-blocks of 8×8 pixels, or as 4×4 sequence of sub-blocks of 4×4pixels. In some implementations, a plurality of sub-blocks of a block ofimage data may contain sub-blocks having different sizes. In someimplementations, a decoder of an image processing system may receiveinput pixel channel samples which are 8 bit data represented in standardRed Green Blue (sRGB (gamma or log)). In some implementations, thedecoder may convert 8 bit sRGB input pixel channel samples to linear13-bit data. In some implementations, encoded image data (e.g., encodedtexel surface data) may be transmitted to an image processing system soas to be directly consumed by a decoder of the image processing system.

The systems and methods described in the present disclosure includeseveral advantages over known systems and methods as follows.

In some implementations, an image processing system can determine acompression mode or a compressing technique best for each sub-block of ablock of image data in terms of energy efficiency based on imagecharacteristics (or data characteristics) of that sub-block, forexample, image quality, a range of values, compression rate, distortionefficiency, thereby achieving high energy efficiency in compressing thewhole block. Similarly, in some implementations, the image processingsystem can determine a compression mode or a compressing technique bestfor each sub-block of a block in terms of memory utilization in an imagestorage based on the image characteristics (or data characteristics) ofthat sub-block, thereby achieving high memory utilization in compressingthe whole block. In some implementations, the systems and methodsdescribed in the present disclosure can provide the following benefitsto a graphics display pipeline for rendering a 3D scene to a 2D screen:(1) texel surface data need to support multiple surfaces with varyingpersistence, thereby requiring efficient memory utilization. The systemsand methods described in the present disclosure can greatly reducedecoder-side storage requirements for texel surface data; and (2) thesystems and methods described in the present disclosure can greatlyreduce power consumption in a high-speed serial interface whentransmitting texel surface data from an encoder-side image processingsystem to a decoder-side image processing system. In this manner, thesystems and methods described in the present disclosure can provide atechnique for compressing image data in a manner of energy efficiencyand efficient memory utilization in an image storage for an eye-trackingapplication, for instance.

In some implementations, an image processing system can determine acompression mode or a compressing technique best for each sub-block of ablock of image data, thereby making the system more optimally adaptiveto the content (e.g., image data) of the block. Compared to conventionalGPU textures in which every block (or sub-block) must be the same size,sub-blocks according to some implementations may be the same size, ormay be different. The “optimal” selection of compression techniques fora block (or sub-block) depends on the content (e.g., image data) of thatblock (or sub-block). As an example, an image of a block containingsub-blocks having different data characteristics may be encoded usingdifferent compression techniques. On the other hand, a perfectly uniformimage of a block may be encoded exactly the same way for every sub-blockof that block. Even an image with lots of structure in some areas butconstant values in other large areas may be encoded exactly the same wayfor every sub-block of that block. Such optimal selection of compressiontechniques can be performed by adapting to the content and independentlydeciding what is optimal on a per-block basis, which can produce adifferently sized compressed representation.

Before turning to the figures, which illustrate certain embodiments indetail, it should be understood that the present disclosure is notlimited to the details or methodology set forth in the description orillustrated in the figures.

FIG. 1 is a block diagram of an embodiment of a system for processingimage data, according to an example implementation of the presentdisclosure. Referring to FIG. 1 , in some implementations, an imageprocessing system 1000 may include one or more encoders 1100, one ormore decryptors 1200, an image data loader 1300, one or more directmemory accesses (DMAs) 1400, one or more encryptors 1500, and one ormore memories 1600. In some implementations, the image processing system1000 may be implemented with a computing system 200 as illustrated inFIG. 2 . For example, the memories 1600 may be implemented with a memory260 in FIG. 2 . The encoders 1100, decryptors 1200, image data loader1300, and encryptors 1500 may be implemented with a processor 210 inFIG. 2 . The one or more encoders 1100 may be implemented with anencoder 7000 in FIG. 7 or an encoder 8000 in FIG. 8 .

Referring to FIG. 1 , in some implementations, the image processingsystem 1000 may receive image data from a network interface device 1700via a high-speed serial interface (e.g., a Mobile Industry ProcessorInterface (MIPI), a Peripheral Component Interconnect Express (PCIe)interface, and so on). In some implementations, the network interfacedevice 1700 may be implemented with a network interface device 230 inFIG. 2 , for example, a network interface device supporting wirelessnetwork connections in which an interface port is a wireless (e.g.,radio) receiver/transmitter (e.g., for any of the IEEE 802.11 protocols,near field communication “NFC”, Bluetooth, ANT, or any other wirelessprotocol). A computing system 200 in FIG. 2 will be described in moredetail in the following sections. Referring back to FIG. 1 , in someimplementations, the decryptors 1200 may decrypt and store the imagedata received from the network interface, in the memories 1600. In someimplementations, the image data loader 1300 may decode and augment imagedata (e.g., data representing images and/or videos) decrypted by thedecryptors 1200 and store the decoded and augmented image data as one ormore frames in the memories 1600. For example, the image data loader1300 can be implemented with the NVIDIA Data Loading Library (DALI). Insome implementations, the image data loader 1300 may generatedescriptors (not shown) for each frame and store the descriptors in thememories 1600 to provide parameters for encoding each frame. Thedescriptors may include but not limited to an address of a buffer (e.g.,an input picture buffer), frame dimensions, and an output address ofencoded frame.

In some implementations, the one or more encoders 1100 may identify aplurality of blocks in a frame based on a size of block (e.g., 16×16pixels). In some implementations, the descriptors may includeinformation on a size of blocks (either fixed size or variable size). Insome implementations, the one or more encoders 1100 may read a block andconvert it to a plurality of blocks having a single channel (singlechannel blocks). For example, each single channel block may correspondto one of a plurality of channels. In some implementations, each channelmay be one of a color channel or an alpha channel. For example, in RGBAcolor space, each channel may be one of three R (red), G (green), B(blue) color channels or A (alpha) channel. In some implementations, theone or more encoders 1100 may read a block in RGBA color space andconvert it to four single channel blocks which correspond to an Rchannel, a G channel, a B channel, and an A channel. In someimplementations, the descriptors may include information for convertinga block to multiple single channel blocks, e.g., color encoding systemsor color spaces such as YUV, CIE XYZ, RGB, and so on.

In some implementations, the one or more encoders 1100 may identify oneor more MIP maps of a frame. In some implementations, the descriptorsmay include information on an MIP map such as number of MIP levels, andresolutions in an associated MIP map. In some implementations, the oneor more encoders 1100 may read a frame of image data and convert it to aplurality of blocks with corresponding MIP levels. For example, eachblock may correspond to one of a plurality of MIP levels (e.g., up to 4MIP levels).

In some implementations, the one or more encoders 1100 may be configuredto read a frame of image data from a buffer (e.g., a decoded picturebuffer), compress the frame and write the compressed data to thememories 1600. In some implementations, the one or more encoders 1100may use one or more compression techniques such as lossless compression,lossy compression, prediction-based compression, or no compression. Insome implementations, the encoders 1100 may read descriptors from thememories 1600 and compress each frame using the descriptors. In someimplementations, the encoders 1100 may update the descriptors while orafter encoding each frame.

In some implementations, the image processing system 100 may be adisplay subsystem for implementing a graphics pipeline. In someimplementations, each of the image processing system 1000 and imageprocessing systems 2000 (see FIG. 1 and FIG. 9 ) may be a displaysubsystem for collaboratively implementing a graphics pipeline. In someimplementations, the DMAs 1400 may be display pipeline DMAs (or graphicspipeline DMAs). For example, the DMAs 1400 may include a left displaypipeline DMA and a right display pipeline DMA. In some implementations,the left and right display pipeline DMAs 1400 may read compressed datafrom the memories 1600 and provide the compressed data to the encryptors1500. In some implementations, the encryptors 1500 may encrypt thecompressed data provided from the DMAs 1400 and transmit the encrypteddata to one or more image processing systems 2000. The image processingsystem 2000 will be described in detail with reference to FIG. 9 . Insome implementations, the encrypted data may be transmitted to the imageprocessing systems via a high-speed display interface, for example, MIPIdisplay serial interface (MIPI DSI). In some implementations, each ofthe encryptors 1500 may transmit encrypted data to a corresponding imageprocessing system 2000. In some implementations, the image processingsystem 2000 may decode, decompress and/or merge the data to provide animage.

FIG. 2 is a block diagram of an embodiment of a computing system,according to an example implementation of the present disclosure.Referring to FIG. 2 , the illustrated example computing system 200includes one or more processors 210 in communication, via acommunication system 240 (e.g., bus), with memory 260, at least onenetwork interface device 230 (or at least one network interfacecontroller) with network interface port for connection to a network (notshown), and other components, e.g., input/output (“I/O”) components 250.Generally, the processor(s) 210 will execute instructions (or computerprograms) received from memory. The processor(s) 210 illustratedincorporate, or are directly connected to, cache memory 220. In someinstances, instructions are read from memory 260 into cache memory 220and executed by the processor(s) 210 from cache memory 220.

In more detail, the processor(s) 210 may be any logic circuitry thatprocesses instructions, e.g., instructions fetched from the memory 260or cache 220. In many implementations, the processor(s) 210 aremicroprocessor units or special purpose processors. The computing device200 may be based on any processor, or set of processors, capable ofoperating as described herein. The processor(s) 210 may be single coreor multi-core processor(s). The processor(s) 210 may be multipledistinct processors.

The memory 260 may be any device suitable for storing computer readabledata. The memory 260 may be a device with fixed storage or a device forreading removable storage media. Examples include all forms ofnon-volatile memory, media and memory devices, semiconductor memorydevices (e.g., EPROM, EEPROM, SDRAM, and flash memory devices), magneticdisks, magneto optical disks, and optical discs (e.g., CD ROM, DVD-ROM,or Blu-Ray® discs). A computing system 200 may have any number of memorydevices 260.

The cache memory 220 is generally a form of computer memory placed inclose proximity to the processor(s) 210 for fast read times. In someimplementations, the cache memory 220 is part of, or on the same chipas, the processor(s) 210. In some implementations, there are multiplelevels of cache 220, e.g., L2 and L3 cache layers.

The network interface device 230 manages data exchanges via the networkinterface (sometimes referred to as network interface ports). Thenetwork interface device 230 handles the physical and data link layersof the OSI model for network communication. In some implementations,some of the network interface controller's tasks are handled by one ormore of the processor(s) 210. In some implementations, the networkinterface device 230 is part of a processor 210. In someimplementations, a computing system 200 has multiple network interfacescontrolled by a single device 230. In some implementations, a computingsystem 200 has multiple network interface devices 230. In someimplementations, each network interface is a connection point for aphysical network link (e.g., a cat-5 Ethernet link). In someimplementations, the network interface device 230 supports wirelessnetwork connections and an interface port is a wireless (e.g., radio)receiver/transmitter (e.g., for any of the IEEE 802.11 protocols, nearfield communication “NFC”, Bluetooth, ANT, or any other wirelessprotocol). In some implementations, the network interface device 230implements one or more network protocols such as Ethernet. Generally, acomputing device 200 exchanges data with other computing devices viaphysical or wireless links through a network interface. The networkinterface may link directly to another device or to another device viaan intermediary device, e.g., a network device such as a hub, a bridge,a switch, or a router, connecting the computing device 200 to a datanetwork such as the Internet.

The computing system 200 may include, or provide interfaces for, one ormore input or output (“I/O”) devices. Input devices include, withoutlimitation, keyboards, microphones, touch screens, foot pedals, sensors,MIDI devices, and pointing devices such as a mouse or trackball. Outputdevices include, without limitation, video displays, speakers,refreshable Braille terminal, lights, MIDI devices, and 2-D or 3-Dprinters.

Other components may include an I/O interface, external serial deviceports, and any additional co-processors. For example, a computing system200 may include an interface (e.g., a universal serial bus (USB)interface) for connecting input devices, output devices, or additionalmemory devices (e.g., portable flash drive or external media drive). Insome implementations, a computing device 200 includes an additionaldevice such as a co-processor, e.g., a math co-processor can assist theprocessor 210 with high precision or complex calculations.

The components 250 may be configured to connect with external media, adisplay 270, an input device 280 or any other components in thecomputing system 3000, or combinations thereof. The display 270 may be aliquid crystal display (LCD), an organic light emitting diode (OLED), aflat panel display, a solid state display, a cathode ray tube (CRT), aprojector, a printer or other now known or later developed displaydevice for outputting determined information. The display 270 may act asan interface for the user to see the functioning of the processor(s)210, or specifically as an interface with the software stored in thememory 260.

The input device 280 may be configured to allow a user to interact withany of the components of the computing system 200. The input device 280may be a plurality pad, a keyboard, a cursor control device, such as amouse, or a joystick. Also, the input device 280 may be a remotecontrol, touchscreen display (which may be a combination of the display270 and the input device 280), or any other device operative to interactwith the computing system 200, such as any device operative to act as aninterface between a user and the computing system 200.

FIG. 3 is a representation of a block of image data for performing datacompression, according to an example implementation of the presentdisclosure. FIG. 4 is a table indicating parameters for performing datacompression, according to an example implementation of the presentdisclosure. An encoding method according to some implementations of thepresent disclosure will be described with reference to FIG. 3 and FIG. 4.

In some implementations, image data (e.g., a frame of image data) can berepresented as a plurality of blocks. For example, texture data may bedivided into a plurality of blocks each of which contains 16×16 pixelsas shown in FIG. 3 . In some implementations, a block of image data maybe divided into a plurality of sub-blocks. In some implementations, ablock of 16×16 pixels (e.g., a block shown in FIG. 3 ) may be dividedinto four sub-blocks of 8×8 pixels, for example, four sub-blocks B0(300), B1 (310), B2 (320), B3 (330) as shown in FIG. 3 . In someimplementations, a block of 16×16 pixels may be divided into 16sub-blocks of 4×4 pixels.

In some implementations, the image data loader 1300 may identify blocksof each frame and sub-blocks of each block, and generate accessinformation on the blocks and sub-blocks. In some implementations, theimage data loader 1300 may write the access information to thedescriptors stored in the memories 1600 to provide access informationfor encoding each block or sub-block of a frame.

In some implementations, the encoders 1100 may determine a coding modeper sub-block (or per block). In some implementations, a coding mode persub-block may be one of a discrete cosine transform (DCT) typecompression mode (e.g., 4×4, 8×8, or basis functions, etc.), aprediction-based compression mode, or an uncompressed mode. In someimplementations, the encoders 1100 may compress image data (e.g.,frames) by transforming the image data based on the coding mode. In someimplementations, a result of the transformation may be quantized andHuffman coded. In some implementations, a size (e.g., byte length) ofeach compressed block may be variable (not fixed). In someimplementations, a size (e.g., byte length) of each compressed sub-blockmay be variable (not fixed). In some implementations, a size of eachcompressed sub-block may be indicated in a header so that in response toreceiving the header and data of compressed sub-blocks, an indirectiontable (e.g., a table 1040 in FIG. 10 ) can be generated.

In some implementations, given a target compression ratio, the encoders1100 may determine a coding mode per sub-block based on datacharacteristics of data of that sub-block to achieve a highest overallimage quality at the target compression ratio. In some implementations,the encoders 1100 may determine a coding mode per sub-block based ondata characteristics of that sub-block to minimize a target costfunction which is a function of an image quality and a compression ratioof a sub-block. In some implementations, the encoders 1100 may determinea coding mode per sub-block based on data characteristics of thatsub-block to maximize a target objective function which is a function ofan image quality and a compression ratio of a sub-block. In someimplementations, an image quality can be represented by at least one ofdistortions, degradations, sharpness, noise, contrast, color accuracy,vignetting, exposure accuracy, or artifacts.

In some implementations, a coding mode of a sub-block based on datacharacteristics of that sub-block may be a plan-of-record mode (orpalette mode) which utilizes a fixed number of bits per pixel sample (orper pixel channel sample if each block corresponds to a channel). Insome implementations, to determine a number of bits per pixel sample ina sub-block (e.g., an 8×8 sub-block), the encoders 1100 may determine arange of values in pixel samples within that sub-block to determineendpoints of significance (e.g., a starting endpoint and an endingendpoint) in that sub-block. In some implementations, the encoders 1100may determine a number of points needed within the determined range thatwill meet or minimize a target cost function, by using a rate-distortionalgorithm. In some implementations, the target cost function may be afunction expressing a trade-off between a distortion efficiency and acompression ratio of a sub-block.

In some implementations, the encoders 1100 may determine a number ofpoints per pixel sample by selecting one of 0 bit, 1 bit, 2 bits, 3bits, 4 bits, 6 bits, or 8 bits, which correspond to number of points of1, 2, 4, 8, 16, 64, or 256 (or uncompressed), per pixel sample. In otherwords, there may be 7 candidate plan-of-record modes per sub-block. Forexample, if 3 bits is selected for a plan-of record mode of a sub-block,the encoders 1100 may compress the sub-block such that up to 8 pointscan be used per pixel sample in compressed sub-block. If 8 bits isselected for a plan-of record mode of a sub-block (as a simple or lowcomplexity mode), the encoders 1100 may not perform compression on thatsub-block.

FIG. 4 is a table indicating parameters for performing data compression,according to an example implementation of the present disclosure. Insome implementations, a plan-of-record mode of a sub-block can bedetermined by three parameters—PRECISION, OFFSET, and SCALE as describedin the table in FIG. 4 . In some implementations, PRECISION can berepresented in 3 bits to define a number of bits per pixel sample usedin a given sub-block. For example, values of PRECISION “000”, “001”,“010”, “011”, “100” and “110” can indicate the number of points of 1, 2,4, 8, 16, 64 being used per pixel sample (to represent a pixel sample)in the given sub-block, respectively. In some implementations, thePRECISION value “101” can indicate that the given sub-block isuncompressed (e.g., 256 levels). In some implementations, the PRECISIONvalue “101” can indicate that the given sub-block is uncompressed butthat it is not assumed that SCALE=255 or OFFSET=0. In someimplementations, the PRECISION value “111” can indicate that the givensub-block is transparent such that the sub-block has no data and is notdisplayed.

In some implementations, OFFSET can be represented in 8 bits to define astarting endpoint of significance. In some implementations, OFFSET canrepresent the number of zero values in input pixel samples in the givensub-block. In some implementations, SCALE can be represented in 8 bitsto define a range of input values of pixel samples in the givensub-block. In some implementations, SCALE can represent a dynamic rangeof input values of pixel samples in the given sub-block.

In some implementations, the encoders 1100 may determine the parametersPRECISION, OFFSET and SCALE for each sub-block and compress data of thatsub-block (denoted by E_(Input)) to output compressed data (denoted byE_(Output)) using the following formula

EOutput=Round((EInput−OFFSET)*(2{circumflex over( )}PRECISION−1)/SCALE))  Equation (1),

where Round(x) means the nearest integer to x.

In some implementations, the encoders 1100 may determine, as acompression technique of a sub-block at least one of losslesscompression, lossy compression, prediction-based compression, or nocompression based on image characteristics (or data characteristics) ofthat sub-block (for example, compression ratio, image quality, etc.).For example, one of lossless compression or lossy compression and theircorresponding compression format can be selected as a compressiontechnique and a compression format for a sub-block based on a targetimage quality of that sub-block. If a target compression ratio of asub-block is relatively high, a prediction-based compression and itscorresponding compression format can be used as a compression techniqueand a compression format for that sub-block. In some implementations,information on compression technique and format used for each sub-blockmay be stored in a header embedded in a block (in a manner similar tothat of the header shown in FIG. 5 ).

With the foregoing encoding schemes, each sub-block of the same blockcan be compressed in a different compression mode (e.g., a differentcompression technique and/or a different compression format) based onimage characteristics (or data characteristics) of that sub-block, suchas image quality, a range of values, compression rate, distortionefficiency, and so on. Moreover, when applying this encoding scheme to aplurality of single channel blocks, data of each single channel block(e.g., a block of a color channel or alpha channel) can be compressedindependently so that different channels of the same image block can becompressed in different compression modes, e.g., by optimally selectingbetween different possible compression techniques.

Now, data formats relating to data compressed by encoders according tosome implementations of the present disclosure will be described withreference to FIG. 5 and FIG. 6 .

FIG. 5 is a table indicating a format of a block header as a result ofperforming data compression, according to an example implementation ofthe present disclosure.

In some implementations, each encoded block (e.g., a block of 16×16pixels) may have a header which contain parameters for decoding thatencoded block. In some implementations, the encoders (e.g., the encoders1100 in FIG. 1 , the encoder 7000 in FIG. 7 , the encoder 8000 in FIG. 8) may generate a header of a block while or after encoding data of theblock. For example, each encoded 16×16 pixel block may have a 80-bitheader as shown in FIG. 5 which contains decoding parameters PRECISION,OPAQUE, OFFSET, SCALE for each encoded sub-block corresponding tosub-block B0, B1, B2, or B3 (see FIG. 3 ). Referring to FIG. 5 , in someimplementations, the fields B0_PRECISION, B1_PRECISION, B2_PRECISION andB3_PRECISION for each encoded sub-block corresponding to sub-block B0,B1, B2, or B3 can be defined in the same manner as the field PRECISIONin FIG. 4 . Similarly, the fields B0_OFFSET, B1_OFFSET, B2_OFFSET andB3_OFFSET for each encoded sub-block corresponding to sub-block B0, B1,B2, or B3 can be defined in the same manner as the field OFFSET in FIG.4 . Similarly, the fields B0_SCALE, B1_SCALE, B2_SCALE and B3_SCALE foreach encoded sub-block corresponding to sub-block B0, B1, B2, or B3 canbe defined in the same manner as the field SCALE in FIG. 4 . In someimplementations, the 1-bit field B0_OPAQUE, B1_OPAQUE, B2_OPAQUE andB3_OPAQUE for each encoded sub-block corresponding to sub-block B0, B1,B2, or B3 can be defined such that ‘1’ indicates that block is opaqueand an Alpha (A-channel) decoding is not needed.

In some implementations, a decoder (e.g., one or more decoders 740 inFIG. 10 ) may read the parameters PRECISION, OFFSET and SCALE for eachsub-block of a block from a header of the block (e.g., the block headeras shown in FIG. 5 ) and decode encoded data of that sub-block to outputdecoded data. The decoding scheme according to some implementations willbe described later with reference to FIG. 9 .

FIG. 6 is a representation of image data as a result of performing datacompression, according to an example implementation of the presentdisclosure.

In some implementations, the encoders (e.g., the encoders 1100 in FIG. 1, the encoder 7000 in FIG. 7 , the encoder 8000 in FIG. 8 ) may compressa plurality of single channel blocks and output compressed data of eachsingle channel block. For example, FIG. 6 shows compressed data of firstR-channel block 601 and compressed data of second R-channel block 602.In some implementations, referring to FIG. 6 , as a result ofcompressing single channel blocks each corresponding to one of R, G, B,A channels, the encoder may output compressed data of R-channel blocks(610, 612, 614), compressed data of G-channel blocks (620, 622, 624),compressed data of B-channel block (630, 632, 634), and compressed dataof an A-channel block (640, 642, 644). Each set of compressed data ofsingle channel blocks may include compressed data 610, 620, 630, 640 ofa first MIP level (e.g., MIP level 0), compressed data 612, 622, 632,642 of a second MIP level (e.g., MIP level 1), and compressed data 614,624, 634, 644 of a third MIP level (e.g., MIP level 2).

In some implementations, based on results of compressing a plurality ofsingle channel blocks, the encoders (e.g., a stream merger 770 of theencoder 7000 in FIG. 7 or a packetizer 830 of the encoder 8000) maygenerate a coded frame including (1) a header (not shown; e.g., atexture update header) and (2) a payload including a plurality of dataslices (e.g., data slices 650, 660, 670 in FIG. 6 ). In someimplementations, each data slice may contain a texture update header(not shown) followed by an integer number of coded (compressed) singlechannel blocks from a specified color channel.

In some implementations, the encoders may divide compressed data of aplurality of single channel blocks into a plurality of data slices. Forexample, the encoders may divide a combined portion of compressed data640 of the A-channel blocks, compressed data 610 of R-channel blocks,compressed data 620 of G-channel blocks, and compressed data 630 ofB-channel blocks (which all correspond to the first MIP level) into aplurality of data slices including a data slice 650 and a data slice660. In some implementations, each of the data slice 650 and the dataslice 660 may contain respective portions of compressed A-channel,R-channel, G-channel, B-channel blocks. For example, the data slice 650includes compressed data 652 of A-channel blocks, compressed data 654 ofR-channel blocks, compressed data 656 of G-channel blocks, andcompressed data 658 of B-channel blocks in this order. With this orderof blocks in a data slice of a frame payload, an encoder-side imageprocessing system (e.g., the image processing system 1000) can sendcompressed data of A-channel blocks to a decoder-side image processingsystem (e.g., the image processing system 2000) before sendingcompressed data of RGB-channel blocks. Similarly, the data slice 660includes compressed data 662 of A-channel blocks, compressed data 664 ofR-channel blocks, compressed data 666 of G-channel blocks, andcompressed data 668 of B-channel blocks in this order. In someimplementations, a data slice may include the same number of singlechannel blocks per channel (e.g., each data slice 650, 660 includes 8single channel blocks per channel).

In some implementations, the encoders may divide a combined portion ofcompressed data 612 of A-channel blocks, compressed data 622 ofR-channel blocks, compressed data 632 of G-channel blocks, andcompressed data 642 of B-channel blocks (which all correspond to thesecond MIP level) into a plurality of data slices including a data slice670. In some implementations, the number of single channel blocks in adata slice corresponding to the second MIP level (e.g., 16 singlechannel blocks in the data slice 670 corresponding to MIP level 1) maybe smaller than that of the data slice corresponding to the first MIPlevel (e.g., 32 single channel blocks in the data slice 650 or 660corresponding to MIP level 0). In some implementations, the data slice670 includes compressed data 672 of A-channel blocks, compressed data674 of R-channel blocks, compressed data 676 of G-channel blocks, andcompressed data 678 of B-channel blocks in this order.

Here, the block structure hierarchy is described as channels, slices,portions, and MIP levels. In some implementations, the describedstructure hierarchy indicates a bottom up structure (e.g., blocks toimages) while it indicates a top down structure (e.g., images to blocks)in other implementations.

FIG. 7 is a block diagram of an embodiment of an encoder for compressingimage data, according to an example implementation of the presentdisclosure. Referring to FIG. 7 , in some implementations, an encoder7000 may include a block fetcher 710, a color space convertor 720 (e.g.,YUV to RGB convertor), one or more downscalers 740 (e.g., one or moredyadic downscalers), one or more block encoders 750, a stream merger770, and an interconnect 730. In some implementations, the interconnect730 may be implemented with the communication system 240 as illustratedin FIG. 2 . In some implementations, the interconnect 730 may be aparallel, synchronous, high-frequency communication interface such asAdvanced eXtensible Interface (AXI). In some implementations, theencoder 7000 may be implemented with the computing system 200 in FIG. 2.

Referring to FIG. 7 , in some implementations, the block fetcher 710 maybe a part of a DMA (e.g., the DMAs 1400 in FIG. 1 ) or an interface toreceive block data from the DMA. In some implementations, the blockfetcher 710 may be configured to identify a plurality of blocks in aframe based on a size of block specified in the descriptors, and read orfetch one or more blocks 731 from the memory (e.g., the memories 1600 inFIG. 1 ) via the interconnect 730. The block fetcher 710 may beconfigured to provide one or more blocks as an input 711 to the colorspace convertor 720 or as an input 713 to a downscaler 740 for alphachannel. In some implementations, the block fetcher 710 may convert theone or more blocks to alpha channel blocks and provide the alpha channelblocks as the input 713 to the alpha channel downscaler.

In some implementations, the color space convertor 720 may convert oneor more blocks of image data in one color encoding system or color model(e.g., YUV) to blocks in another color encoding system or color model(e.g., RGB). For example, the color space convertor 720 may convert ablock of image data encoded in YUV to three single channel blocks, forexample, an R-channel block 721, a G-channel block 723, and a B-channelblock 735, and provide the converted single channel blocks to thecorresponding downscalers 740. The input color space is not limited toYUV and may be native display primaries (in which case no colorconversion may be needed), RGB (e.g., AdobeRGB), or an HD video (e.g.,Rec709). The output color space is not limited to RGB and may be otherthan RGB (e.g., CIE XYZ). In some implementations, the output colorspace is native display primaries (e.g., Digital Cinema Initiatives(DCI)-P3). In some implementations, input image data may be associatedwith motion vectors, for example, motion vector (x, y). In someimplementations, the motion vector (x, y) may have a 16-bit depth suchthat x*y is a close approximation of depth. In some implementations, themotion vector may have a 8-bit log depth in N4.4 format, for example,depth=2^((d/16)) which can give the depth in the range 1 to 62757.5 inmultiplicative steps of 2{circumflex over ( )}( 1/16)=1.044 (2% relativeerror max).

In some implementations, the block fetcher 710 may identify one or moreMIP maps of a frame based on MIP map information (e.g., MIP levels andresolutions) included in the descriptors, and provide one or more blocksof corresponding MIP levels as the input 711 to the color spaceconvertor 720. In some implementations, the color space convertor 720may convert one or more blocks of the corresponding MIP levels in onecolor encoding system or color model (e.g., YUV) to blocks in anothercolor encoding system or color model (e.g., RGB).

Referring to FIG. 7 , in some implementations, each of the downscalers740 may be a dyadic downscaler configured to downscale a single channelblock down to the ratio of 1, ½, ⅛, etc. (denoted by “/1”, “/2”, “/4”,“/8” downscales, respectively). The present disclosure, however, is notlimited thereto; in some implementations, the downscalers 740 maysupport downscales other than the dyadic downscales. In someimplementations, each of the downscalers 740 may selectively determine adownscale for blocks of different single channels or different MIPlevels. For example, the downsclalers 740 may select “/1” downscale forRGB single channels of MIP level 0. The downsclalers 740 may selectdownscales other than “/1” for blocks of higher MIP levels (e.g., MIPlevels 1, 2, . . . ). In some implementations, the downscalers 740 mayselectively determine a downscale for blocks with different samplingratios. For example, for blocks sampled using the sampling ratio of GRB4:2:0 in which there is no R-channel blocks of MIP level 0 nor B-channelblocks of MIP level 0, the downscalers 740 may select “/1” for G-channelblocks, “/2” for R-channel blocks, and “/2” for B-channel blocks. Thesampling ratio is not limited to GRB 4:2:0 and may be other ratios, forexample, RGB 4:4:4.

In some implementations, each of the one or more block encoders 750 mayreceive, from the downscalers 740, a corresponding downscaled colorchannel block (e.g., a downscaled R-channel block 741, a downscaledG-channel block 743, a downscaled B-channel block 745), and encode thecorresponding downscaled color channel block to be provided as acorresponding input (an encoded R-channel block 751, an encodedG-channel block 753, an encoded B-channel block 755) to the streammerger 770. Similarly, the block encoder 760 for alpha channel mayreceive, from the downscaler 740, a corresponding downscaled alphachannel block 747, and encode the downscaled alpha channel block to beprovided as an input (an encoded A-channel block 757) to the streammerger 770.

In some implementations, each of the block encoders 750 and blockencoder 760 may determine parameters PRECISION, OFFSET and SCALE foreach sub-block of the corresponding downscaled block and compress dataof that sub-block to output compressed data using Equation (1) asdescribed above. In some implementations, each of the block encoders 750and block encoder 760 may determine, as a compression technique of asub-block at least one of lossless compression, lossy compression,prediction-based compression, or no compression based on imagecharacteristics (or data characteristics) of that sub-block (forexample, compression ratio, image quality, etc.). In someimplementations, each block encoder may store information on compressiontechnique and format used for each sub-block of the corresponding blockin a header embedded in that block (in a manner similar to that of theheader shown in FIG. 5 ).

In some implementations, the stream merger 770 may receive the encodedcolor channel blocks (e.g., blocks 751, 753, 755) and the encoded alphachannel blocks (e.g., block 757), merge the received blocks, andprovide, via the interconnect 730, an encoder output stream 771 forother image processing to, for example, encryptors (e.g., the encryptors1500) or decoder-side image processing systems (e.g., the imageprocessing systems 2000). In some implementations, the stream merger 770may generate a coded frame including (1) a header (not shown; e.g., atexture update header) and (2) a payload including a plurality of dataslices (e.g., data slices 650, 660, 670 in FIG. 6 ). In someimplementations, the stream merger 770 may generate each data slice bydividing encoded data of received blocks (e.g., blocks 751, 753, 755,757) into a plurality of data slices, as described above with referenceto FIG. 6 .

FIG. 8 is a block diagram of another embodiment of an encoder forcompressing image data, according to an example implementation of thepresent disclosure. Referring to FIG. 8 , in some implementations, anencoder 8000 may include at least one of a finite state machine (FSM)806, one or more block fetcher 808, one or more upscalers 810, a colormatrix 812, a 3D lookup table (LUT) 814, one or more downscalers 816, agamma corrector 818, a sub-res memory 820, a data merger 822, a stats orerrors calculator 824, a memory writer 826, a palette-quantizer 828,and/or packetizer 830. In some implementations, the encoder 8000 may beimplemented with the computing system 200 in FIG. 2 . In someimplementations, encoder 8000 may be configured read/write from/to amemory (e.g., the memories 1600 in FIG. 1 ) using a memory mappedcommunication system 804. In some implementations, the memory mappedcommunication system 804 may be implemented with a memory mappedcommunication between a master device and a slave device in a parallelhigh-performance, synchronous communication interface (e.g., AdvancedeXtensible Interface (AXI) crossbar).

Referring to FIG. 8 , in some implementations, the encoder 8000 may beconfigured to read/write from/to a memory (e.g., the memories 1600 inFIG. 1 ) by issuing read/write commands over a communication system8100. In some implementations, the communication system 8100 may beimplemented with the communication system 240 in FIG. 2 . In someimplementations, the communication system 8100 may be implemented with aparallel high-performance, synchronous communication interface (e.g.,Advanced eXtensible Interface (AXI)). In some implementations, the FSM806 of the encoder 8000 may be configured to access or update controland status registers (CSRs—not shown) of peripheral devices which areconnected to a bus for a low bandwidth control access, for example, ARMAdvanced Peripheral Bus (APB—not shown).

In some implementations, the FSM 806 of the encoder 8000 may beconfigured to read descriptors 801 from the memory and parse them toobtain information included therein. In some implementations, thedescriptors may include but not limited to an address of a buffer (e.g.,an input picture buffer), frame dimensions, and an output address ofencoded frame, information on a size of blocks (either fixed size orvariable size, and/or information for converting a block to multiplesingle channel blocks, e.g., color encoding systems or color spaces suchas YUV, CIE XYZ, RGB, and so on. In some implementations, the FSM 806may control the flow of encoding image blocks based on informationincluded in the descriptors. In some implementations, the FSM 806 maydetermine a sequence of encoding process based on information includedin the descriptors. In some implementations, the FSM 806 may determine astate based on information included in the descriptors, select aprocessing component (among the components 808, 810, 812, 814, 816, 818,820, 822, 824, 828, 830, for example), and send control information 805to cause the selected processing component to perform (image dataencoding) processing. In some implementations, after the selectedprocessing component completes the processing, the FSM 806 may updatethe descriptors with information relating to the current state and otherupdate information, by writing the updated descriptors 802 to thememory.

For example, in accordance with the control by the FSM 806, the blockfetcher 808 may fetch a block of image data 803 based on addressinformation of the block in the descriptors. The upscaler 810 mayupscale UV image texture of the block, if necessary. The FSM 806 mayconvert the block of image data from one color encoding system or colormodel (e.g., YUV) to another color encoding system or color model (e.g.,RGB) using the color matrix 812 and/or the 3D LUT 814. For example, theblock of image data encoded in YUV may be converted to three singlechannel blocks. The downscaler 816 may downscale a single channel blockdown to, for example, “/1”, “/2”, “/4”, “/8” downscales. The image dataof the block may be optimized by the gamma corrector 818 and/or thesub-res memory 820. In some implementations, the sub-res memory 820 mayapply sub-pixel resolution method to the image data of the block. Insome implementations, with the sub-res memory, downscaling the imagescan be done recursively, for example, /1 downscale is used to generate/2downscale, which is then used to generate/4 downscale, which is thenused to generate/8 downscale, etc. The data merger 822 may mergeprocessing results 807, 809 into a block of image data. The states orerrors calculator 824 may check states or errors in the block of imagedata based on information included in the descriptors. In response todetermining that there is no errors in the block of image data, thestates or errors calculator 824 may send control information 813 tocause the memory writer 826 to write the block (as data 804) to thememory, and send control information 811 to cause the palette-quantizer828 to perform encoding on the block. In response to determining thatthere is any errors in the block of image data, the FSM 806 may stop theencoding processing and exit, or perform further processing to fix theerrors and resume the encoding processing. Here, errors mean differencesbetween the original pixel values and their compressed version. In someimplementations, these differences are expected, and the goal of thecompression system is to make them as small as possible, but not toeliminate them because eliminating errors would produce losslesscompression, which does not reduce the block size enough.

In some implementations, a palette-quantizer 828 may receive a block ofimage data and determine parameters PRECISION, OFFSET and SCALE for eachsub-block of that image data block and compress data of that sub-blockto output compressed data using Equation (1) as described above. In someimplementations, the palette-quantizer 828 may store information oncompression technique and format used for each sub-block of thecorresponding block in a header embedded in that block (in a mannersimilar to that of the header shown in FIG. 5 ).

In some implementations, the packetizer 830 may receive the encoded orcompressed blocks from the palette-quantizer 828 and generate one ormore packets which may be a coded frame including (1) a header (notshown; e.g., a texture update header) and (2) a payload including aplurality of data slices (e.g., data slices 650, 660, 670 in FIG. 6 ).In some implementations, the packetizer 830 may generate each data sliceby dividing encoded or compressed data of received blocks into aplurality of data slices, as described above with reference to FIG. 6 .After generating one or more packets 817, the packetizer 830 may sendcontrol information 817 to cause the memory writer 826 to write thepackets (as data 804) to the memory.

FIG. 9 is a block diagram of an embodiment of a system for processingimage data, according to an example implementation of the presentdisclosure. Referring to FIG. 9 , in some implementations, an imageprocessing system 2000 may include one or more encoders 940, one or moredecryptors 910, a parser 920, one or more ingress direct memory accesses(DMAs) 930, one or more texture caches including a texture cache forcolor channel blocks 962 and a texture cache for alpha channel blocks964, a display control processor (DCP) 950, one or more memories 980including, for example, mesh storage 982, lookup tables storage 984,color textures storage 986, alpha textures storage 988, and aninterconnect 960. In some implementations, the image processing system2000 may be implemented with a computing system 200 as illustrated inFIG. 2 . For example, the memories 980 may be implemented with a memory260 in FIG. 2 . The decoders 940, decryptors 910, parser 920, and DCP950 may be implemented with a processor 210 in FIG. 2 . The one or moretexture caches 962, 964 may be implemented with a cache 220 in FIG. 2 .The interconnect 960 may be implemented with a communication system 240in FIG. 2 . In some implementations, the interconnect 960 may beimplemented with one or more network on chips (NoC).

In some implementations, the image processing system 2000 may receivedata from an image processing system (e.g., an image processing system1000 in FIG. 1 ), and decode, decompress and/or merge the data toprovide an image. Referring to FIG. 9 , in some implementations, theimage processing system 2000 may receive an encoder output stream (e.g.,compressed data of a plurality of blocks) from the image processingsystem 1000 via a high-speed serial interface (e.g., a Mobile IndustryProcessor Interface (MIPI), a Peripheral Component Interconnect Express(PCIe) interface, and so on). In some implementations, the decryptors910 may decrypt the received encoder output stream. In someimplementations, the parser 920 may parse and store the decryptedencoder output stream, in the memories 980 via the interconnect 960. Insome implementations, the parser 920 may perform an ingress process.That is, from the decrypted encoder output stream, the parser 920 mayidentify mesh data, color textures data (e.g., a plurality of colorchannel blocks) and alpha textures data (e.g., a plurality of alphachannel blocks) and store them into the mesh storage 982, the colortextures storage 986 and the alpha textures storage 988, respectively.In some implementations, from the decrypted encoder output stream, theparser 920 may identify (or calculate or build or generate) addressinformation of each encoded block (e.g., an indirection as shown in FIG.10 ) and store the address information in the lookup tables storage 984as a part of the ingress process. In some implementations, the parser920 may store the address information in an indirection table (e.g., theindirection table 1040 in FIG. 10 ) in the lookup tables storage 984. Insome implementations, while reading a plurality of encoded blocks fromthe memories 980, the decoder 940 (or DMAs of the decoder 940) may usethe coordinates (e.g., x, y), channel, and/or MIP level of a block tocompute a “lookup” address of that block. In some implementations, thedecoder 940 may compute a lookup address based on a lookup table (e.g.,the indirection table 1040 in FIG. 10 ), which stores data containing anaddress for a compressed texture data, for example.

In some implementations, the image processing system 2000 may be adisplay subsystem for implementing a graphics pipeline. In someimplementations, the ingress DMAs 930 may be display pipeline DMAs (orgraphics pipeline DMAs). For example, the ingress DMAs 930 may include aleft display pipeline DMA and a right display pipeline DMA. In someimplementations, the left and right display pipeline DMAs may readcompressed data from the memories 980 and write the compressed data tothe memory 980. In some implementations, the decoder 940 may decode thecompressed data stored in the memory 980. In some implementations, thedecoder may have its own DMA to access blocks in the memory 980 asneeded. In some implementations, the decoder may access blocks in anon-raster order at different MIP levels based on a head position.

In some implementations, in response to receiving display commands(e.g., MIPI display commands; example commands are shown below inTable 1) sent by the image processing system 1000 and receiving decodeddata from the decoder 940, the DCP 950 may provide to one or moredisplays (e.g., displays 992, 994 as shown in FIG. 9 ) image data (orvideo data) including pixel values and synchronization information(e.g., Vsync, Hsync, data enable, and the pixel clock), for example. Insome implementations, the DCP 950 may be tightly coupled to the displaysincluding display components such as light emitting diodes (LED),liquid-crystal displays (LCD), or liquid crystal on silicon (LCOS),which can form the image to be displayed for the users to see.

In some implementations, the texture cache 962 may store color channelblocks so that future requests for color channel blocks can be servedfaster without accessing the color textures storage 986. In someimplementations, the decoders 940 may obtain address information of aparticular color channel block by accessing an indirection table (e.g.,the indirection table 1040 in FIG. 10 ) stored in the lookup tablesstorage 984. In some implementations, with the address information, thedecoders 940 may send a request 941 for a particular color channel block(e.g., a single channel block corresponding to one of R-channel,G-channel, or B-channel) to the texture cache 962. In someimplementations, the request 941 may include one or more of a textureidentifier (e.g., 8 bit identifier), a channel, an MIP level, or Uvalues and V values (e.g., 16 U values and 16 V values). A cache hitoccurs when the requested particular color channel block can be found inthe cache, while a cache miss occurs when it cannot. In someimplementations, when a cache miss occurs, the texture cache 962 mayidentify information on the missed color channel block including one ormore of a block identifier (e.g., 2 bit identifier for identifying an8×8 block), an identifier of a cache entry (or slot), a textureidentifier (e.g., 8 bit identifier), an MIP level, or a U value (e.g., 7bit value) and a V value (e.g., 7 bit value), and look up addressinformation of the missed block on lookup tables (e.g., the lookuptables storage 984) using the identified information. In someimplementations, upon a cache miss, the texture cache 962 may removesome cache entry in order to make room for newly retrieved color channelblock(s), send a request 943 for the particular color channel block tothe color textures storage 986, and receive and store the requestedblock 944 to serve it to the decoders 940. In some implementations, therequest 943 may include address information of the missed block, and therequested block 944 may include at least one of color data (e.g., RGBdata of an 8×8 block) or an MIP level (e.g., depth information of an 8×8block).

Similarly, in some implementations, the decoders 940 may obtain addressinformation of a particular alpha channel block by accessing anindirection table stored in the lookup tables storage 984. In someimplementations, with the address information, the decoders 940 may senda request 945 for a particular alpha channel block to the texture cache964. A cache hit occurs when the requested particular alpha channelblock can be found in the cache, while a cache miss occurs when itcannot. Upon a cache miss, the texture cache 964 may remove some cacheentry in order to make room for newly retrieved alpha channel block(s),send a request 947 for the particular alpha channel block to the alphatextures storage 988, and receive and store the requested block 948 toserve it to the decoders 940. In some implementations, the requestedblock 948 may include at least one of alpha channel data (e.g., alphachannel data of an 8×8 block) or alpha channel flags (e.g., informationto specify whether alpha blending should be used).

In some implementations, the one or more decoders 940 may readparameters PRECISION, OFFSET and SCALE for each sub-block of a blockfrom a header of the block (e.g., the block header as shown in FIG. 5 )and decode encoded data of that sub-block (denoted by D_(Input)) tooutput decoded data (denoted by D_(Output)) to the DCP 950 using thefollowing formula:

D _(Output)=OFFSET+((D _(Input)*SCALE)/(2 PRECISION−1))  Equation (2)

In some implementations, a decoder may read the parameter OPAQUE foreach sub-block of a block from a header of the block and determine,based on a value of the OPAQUE field, whether that encoded sub-blockneeds an A-channel decoding. In some implementations, in response todetermining that an encoded sub-block needs an A-channel decoding (forexample, the OPAQUE value is ‘0’), the decoder may perform an A-channeldecoding on a given sub-block.

In some implementations, the decoders 940 may obtain information oncompression technique and format used for each sub-block of a block,from a header embedded in that block, and decode, based on the obtainedinformation, encoded data of that sub-block to output decoded data tothe DCP 950. For example, the information obtained from the header mayinclude (1) one of lossy compression, prediction-based compression, orno compression as a compression technique of a sub-block and (2) itscorresponding compression format.

Referring to FIG. 9 , in some implementations, assuming an encoderoutput stream (e.g., compressed data of a plurality of blocks) isreceived from the image processing system 1000 via a MIPI interfaces, anexample sequence of MIPI display commands (sent by the image processingsystem 1000 to the image processing system 2000) and operations carriedout by the system 2000 (e.g., by the DCP 950 of the system 2000) inresponse to the commands are shown below in Table 1.

TABLE 1 Sequence Number Commands Operations in receiving side (e.g.,system 2000) 1 VSYNC (time) Vertical synchronization 2 Surface 1 SetupSet up a frame buffer (memory) address for surface texture of MIP level1 3 Surface 1 Texture Receive blocks of surface texture of MIP level 1Update 4 Surface 2 Setup Set up a frame buffer address for surfacetexture of MIP level 2 5 Surface 2 Texture Receive blocks of surfacetexture of MIP level 2 Update 6 Surface 2 Texture Receive blocks ofsurface texture of MIP level 2 Update 7 DONE 1 Complete processing ofsurface texture of MIP level 1; release frame buffer for surface textureof MIP level 1 8 Surface 2 Texture Receive blocks of surface texture ofMIP level 2 Update 9 DONE 2 Complete processing of surface of MIP level2; release frame buffer for surface texture of MIP level 2 . . . . . . NVSYNC (time) Vertical synchronization N + 1 Surface 2 Texture Receiveblocks of surface texture of MIP level 2 (reuse Update frame bufferaddress already set up before; in other words, this command should bepreceded by a Surface 2 Setup) N + 2 Surface 3 Setup Set up frame bufferaddress for surface texture of MIP level 3 N + 3 Surface 3 TextureReceive blocks of surface texture of MIP level 3 Update N + 4 DONE 3Complete processing of surface of MIP level 3; release frame buffer forsurface texture of MIP level 3 N + 5 Surface 2 Texture Receive blocks ofsurface texture of MIP level 2 Update N + 6 DONE 2 Complete processingof surface of MIP level 2

In some implementations, each surface may be updated without any overlapwith updates of other surfaces. For example, for MIP level i, thesequence of “Surface i Setup”, “Surface i Texture Update”, “DONE i” maybe performed without any intervening operations.

FIG. 10 is an indirection table for performing decoding of image data,according to an example implementation of the present disclosure.

Referring to FIG. 10 , an image processing system (e.g., the imageprocessing system 2000 in FIG. 9 ) may receive an encoder output streamfrom an encoder-side image processing system (e.g., the image processingsystem 1000 in FIG. 9 ) and store the encoder output stream in a memory(e.g., the color textures storage 986 or the alpha textures storage 988in FIG. 9 ). In some implementations, the encoder output stream mayinclude a coded frame containing a data slice 1020 as a payload of thecoded frame. In some implementations, the data slice may includecompressed data of single channel blocks corresponding to differentchannels (e.g., a color channel or an alpha channel). In someimplementations, the data slice 1020 may include compressed data ofsingle channel blocks of a first MIP level (e.g., MIP level 1)corresponding a first channel (e.g., R-channel blocks 1021, 1022),compressed data of single channel blocks of the first MIP levelcorresponding a second channel (e.g., G-channel blocks 1023, 1024),compressed data of single channel blocks of the first MIP levelcorresponding a third channel (e.g., B-channel blocks 1025, 1026),compressed data of single channel blocks of a second MIP level (e.g.,MIP level 2) corresponding the first channel (e.g., R-channel block1027), compressed data of single channel blocks of the second MIP levelcorresponding the second channel (e.g., G-channel block 1028) andcompressed data of single channel blocks of the second MIP levelcorresponding the third channel (e.g., B-channel block 1029), in thisorder. In some implementations, the size (e.g., byte length) of eachsingle channel block may be variable. In some implementations, the sizeof a block may be embedded with the block in a header thereof (notshown).

In some implementations, an image processing system (e.g., the parser920 or the DMA 730 of the image processing system 2000) may identify orextract the size of each block (e.g., each single channel block1021-1029 in FIG. 10 ) from a header thereof while reading the blocksfrom the memory, and generate an indirection table 1040 on the fly basedon the identified or extracted block sizes. In some implementations, theimage processing system 2000 may identify or extract the size of eachblock from a header thereof while storing the blocks in the memory. Insome implementations, the image processing system 2000 may store theindirection table 1040 in a memory (e.g., the lookup tables 984 in FIG.9 ). In some implementations, the indirection table 1040 may include aplurality of sub-tables for blocks of respective channels (e.g., asub-table 1042 for R-channel blocks, a sub-table 1044 for G-channelblocks, a sub-table 1046 for B-channel blocks in FIG. 10 ). In someimplementations, the indirection table may include a plurality ofentries for corresponding blocks. In some implementations, each entry ofthe indirection table may store address information of the correspondingblock in the memory, such as block index, offset (in the payload) and/orlength of the block. In some implementations, each entry of theindirection table may store information for decoding sub-blocks of thecorresponding block. For example, an entry of the indirection table maystore a block header of the corresponding block similar to that shown inFIG. 5 .

In some implementations, the indirection table may be generated or builtor populated as the image processing system 200 identifies or extractsthe size of each block while reading the blocks of the data slice fromthe memory in the order of storing them (e.g., from the block 1021 tothe block 1029 in FIG. 10 ). For example, as shown in FIG. 10 , theimage processing system 200 identifies or extracts byte lengths of theblocks 1021 to 1029 as 88, 50, 30, 40, 30, 40, 50, 50, 50. Assuming thestarting address of the block 1021 in the memory is zero, as itidentifies or extracts size information of the blocks 1021, 1022, theimage processing system 200 may first calculate block index and offsetof the blocks 1021, 1022 based on the sizes of those blocks, and writethe block index and offset to the sub-table 1042. Then, as it identifiesor extracts size information of the blocks 1023, 1024 of G-channel, theimage processing system 200 may locate the sub-table 1044 (indicated byarrow 1051 in FIG. 10 ), calculate block index and offset of the blocks1023, 1024 based on the sizes of those blocks, and write the block indexand offset to the sub-table 1044. Then, as it identifies or extractssize information of the blocks 1025, 1026 of B-channel, the imageprocessing system 200 may locate the sub-table 1046 (indicated by arrow1053 in FIG. 10 ), calculate block index and offset of the blocks 1025,1026 based on the sizes of those blocks, and write the block index andoffset to the sub-table 1046. Then, as it identifies or extracts sizeinformation of the block 1027 of R-channel, the image processing system200 may refer back to the sub-table 1042 (indicated by arrow 1055 inFIG. 10 ), calculate block index and offset of the block 1027 based onthe size of that block, and write the block index and offset to thesub-table 1042.

In some implementations, a device for compressing image data may includeone or more processors (e.g., the processor 210 in FIG. 2 ), coupled tomemory (e.g., the memory 260 in FIG. 2 or the memories 1600 in FIG. 1 ).The one or more processors may be configured to identify a plurality ofsub-blocks of a block of image data (e.g., sub-blocks B0, B1, B2, B3 inFIG. 3 ). The one or more processors may be further configured toidentify a first data characteristic of data of a first sub-block of theblock of image data (e.g., a range of values in pixel samples withinthat sub-block) and a second data characteristic of data of a secondsub-block of the block of image data (e.g., a range of values in pixelsamples within that sub-block). The one or more processors may befurther configured to determine a first compression technique (e.g., acompression with PRECISION, OFFSET and SCALE in FIG. 4 ) based at leaston the first data characteristic of the first sub-block. The one or moreprocessors may be further configured to determine, based at least on thesecond data characteristic of the second sub-block, a second compressiontechnique (e.g., a compression with PRECISION, OFFSET and SCALE in FIG.4 ) that is different from the first compression technique. The one ormore processors (e.g., via the block encoder 750 in FIG. 7 or thepalette-quantizer 828 in FIG. 8 ) may be further configured to compressthe first sub-block using the first compression technique and the secondsub-block using the second compression technique.

In some implementations, the data characteristic of each of the firstand second sub-blocks may be at least one of an image quality, a rangeof values (e.g., a range of values in pixel samples within a sub-block),a compression rate, or a distortion efficiency. The processor may befurther configured to determine a range of values within each of thefirst and second sub-blocks such that the range of values satisfy atarget cost function of (1) a compression rate and (2) a distortionefficiency (e.g., a function expressing a trade-off between a distortionefficiency and a compression ratio of a sub-block).

In some implementations, each of the first and second compressiontechniques may be one of lossless compression, lossy compression,prediction-based compression, or no compression. In someimplementations, the compressed first sub-block may have a compressionformat different from a compression format of the compressed secondsub-block. For example, one of lossless compression or lossy compressionand their corresponding compression format can be selected as acompression technique and a compression format for a sub-block based ona target image quality of that sub-block. If a target compression ratioof a sub-block is relatively high, a prediction-based compression andits corresponding compression format can be used as a compressiontechnique and a compression format for that sub-block.

In some implementations, the block of image data may correspond to oneof a plurality of channels. Each channel may be one of a color channelor an alpha channel. For example, referring to FIG. 7 , the color spaceconvertor 720 may convert a block of image data encoded in YUV to threesingle channel blocks, for example, an R-channel block 721, a G-channelblock 723, and a B-channel block 735.

In some implementations, the one or more processors may be furtherconfigured to generate a plurality of first compressed blocks (e.g.,compressed blocks 610, 620, 630, 640 in FIG. 6 ) by compressing aplurality of first blocks. Each block of the plurality of first blocksmay correspond to one of the plurality of channels (e.g., compressedR-channel block 610, compressed G-channel block 620, compressedB-channel block 630, and compressed A-channel block 640). The one ormore processors may be further configured to divide the plurality offirst compressed blocks into a plurality of first data slices (e.g.,data slices 650, 660 in FIG. 6 ). Each of the plurality of first dataslices may include a plurality of portions, such that each portionincludes compressed blocks corresponding to a respective one of theplurality of channels (e.g., the data slice 650 includes compressed data652 of A-channel blocks, compressed data 654 of R-channel blocks,compressed data 656 of G-channel blocks, and compressed data 658 ofB-channel blocks in FIG. 6 ).

In some implementations, the one or more processors may be furtherconfigured to generate a plurality of second compressed blocks (e.g.,compressed blocks 612, 622, 632, 642 in FIG. 6 ) by compressing aplurality of second blocks. Each block of the plurality of second blocksmay correspond to one of the plurality of channels (e.g., compressedR-channel block 612, compressed G-channel block 622, compressedB-channel block 632, and compressed A-channel block 642). The one ormore processors may be further configured to divide the plurality ofsecond compressed blocks into a plurality of second data slices (e.g.,data slice 670 in FIG. 6 ). Each of the plurality of second data slicesmay include a plurality of portions, such that each portion includescompressed blocks corresponding to a respective one of the plurality ofchannels (e.g., the data slice 670 includes compressed data 672 ofA-channel blocks, compressed data 674 of R-channel blocks, compresseddata 676 of G-channel blocks, and compressed data 678 of B-channelblocks in FIG. 6 ). The plurality of first blocks may have a first MIPlevel (e.g., MIP level 0 of the compressed blocks 610, 620, 630, 640 inFIG. 6 ) different from a second MIP level of the plurality of secondblocks (e.g., MIP level 1 of the compressed blocks 612, 622, 632, 642 inFIG. 6 ).

In some implementations, the one or more processors may be furtherconfigured to generate, based on the plurality of first data slices(e.g., the data slice 1020 in FIG. 10 ), an indirection table (e.g., theindirection table 1040 in FIG. 10 ) indicating address information(e.g., offset and length fields in the indirection table 1040 in FIG. 10) of compressed blocks in the plurality of first data slices.

In some implementations, a non-transitory computer readable mediumstoring program instructions for causing one or more processors (e.g.,the processor 210 in FIG. 2 ) to identify a plurality of sub-blocks of ablock of image data (e.g., sub-blocks B0, B1, B2, B3 in FIG. 3 ). Theone or more processors may be further caused to identify a first datacharacteristic of data of a first sub-block of the block of image data(e.g., a range of values in pixel samples within that sub-block) and asecond data characteristic of data of a second sub-block of the block ofimage data (e.g., a range of values in pixel samples within thatsub-block). The one or more processors may be further caused todetermine a first compression technique (e.g., a compression withPRECISION, OFFSET and SCALE in FIG. 4 ) based at least on the first datacharacteristic of the first sub-block. The one or more processors may befurther caused to determine, based at least on the second datacharacteristic of the second sub-block, a second compression technique(e.g., a compression with PRECISION, OFFSET and SCALE in FIG. 4 ) thatis different from the first compression technique. The one or moreprocessors (e.g., via the block encoder 750 in FIG. 7 or thepalette-quantizer 828 in FIG. 8 ) may be further caused to compress thefirst sub-block using the first compression technique and the secondsub-block using the second compression technique. In someimplementations, the data characteristic of each of the first and secondsub-blocks may be at least one of an image quality, a range of values(e.g., a range of values in pixel samples within a sub-block), acompression rate, or a distortion efficiency.

FIG. 11 is a flow chart illustrating a process to compress a block ofimage data, according to an example implementation of the presentdisclosure. In some implementations, the method includes identifying, bya processor, a plurality of sub-blocks of a block of image data (1102).The method can include identifying, by the processor, a first datacharacteristic of data of a first sub-block of the block of image dataand a second data characteristic of data of a second sub-block of theblock of image data (1104). The method can include determining, by theprocessor, a first compression technique based at least on the firstdata characteristic of the first sub-block (1106). The method caninclude determining a second compression technique by the processorbased at least on the second data characteristic of the second sub-block(1108). The method can include compressing, by the processor, the firstsub-block using the first compression technique and the second sub-blockusing the second compression technique (1110).

In further details of 1102, and in some implementations, a processor(e.g., the processor 210 in FIG. 2 ) may identify a plurality ofsub-blocks of a block of image data (e.g., sub-blocks B0, B1, B2, B3 inFIG. 3 ).

In further details of 1104, and in some implementations, the processormay identify a first data characteristic (e.g., a range of values inpixel samples within that sub-block) of data of a first sub-block of theblock of image data and a second data characteristic (e.g., a range ofvalues in pixel samples within that sub-block) of data of a secondsub-block of the block of image data. In some implementations, the datacharacteristic of each of the first and second sub-blocks may be atleast one of an image quality, a range of values, a compression rate, ora distortion efficiency. In some implementations, the processor maydetermine a range of values within each of the first and secondsub-blocks such that the range of values satisfy a target cost functionof (1) a compression rate and (2) a distortion efficiency (e.g., afunction expressing a trade-off between a distortion efficiency and acompression ratio of a sub-block).

In some implementations, the block of image data may correspond to oneof a plurality of channels. Each channel may be one of a color channelor an alpha channel. For example, referring to FIG. 7 , the color spaceconvertor 720 may convert a block of image data encoded in YUV to threesingle channel blocks, for example, an R-channel block 721, a G-channelblock 723, and a B-channel block 735.

In further details of 1106, and in some implementations, the processormay determine a first compression technique (e.g., a compression withPRECISION, OFFSET and SCALE in FIG. 4 ) based at least on the first datacharacteristic of the first sub-block (e.g., a range of values in pixelsamples within that sub-block).

In further details of 1108, and in some implementations, the processormay determine, based at least on the second data characteristic of thesecond sub-block (e.g., a range of values in pixel samples within thatsub-block), a second compression technique (e.g., a compression withPRECISION, OFFSET and SCALE in FIG. 4 ). In some implementations, thesecond compression technique determined based on the second datacharacteristic that is different from the first data characteristic, isdifferent from the first compression technique. In some implementations,each of the first and second compression techniques may be one oflossless compression, lossy compression, prediction-based compression,or no compression. In some implementations, the compressed firstsub-block may have a compression format different from a compressionformat of the compressed second sub-block. For example, one of losslesscompression or lossy compression and their corresponding compressionformat can be selected as a compression technique and a compressionformat for a sub-block based on a target image quality of thatsub-block. If a target compression ratio of a sub-block is relativelyhigh, a prediction-based compression and its corresponding compressionformat can be used as a compression technique and a compression formatfor that sub-block.

In further details of 1110, and in some implementations, the processor(e.g., via the block encoder 750 in FIG. 7 or the palette-quantizer 828in FIG. 8 ) may compress the first sub-block using the first compressiontechnique (e.g., a compression with PRECISION, OFFSET determined basedon the data characteristics of the first sub-block) and the secondsub-block using the second compression technique (e.g., a compressionwith PRECISION, OFFSET determined based on the data characteristics ofthe second sub-block).

In some implementations, the method may further include generating aplurality of first compressed blocks (e.g., compressed blocks 610, 620,630, 640 in FIG. 6 ) by compressing a plurality of first blocks. Eachblock of the plurality of first blocks may correspond to one of theplurality of channels (e.g., compressed R-channel block 610, compressedG-channel block 620, compressed B-channel block 630, and compressedA-channel block 640). The method may further include dividing theplurality of first compressed blocks into a plurality of first dataslices (e.g., data slices 650, 660 in FIG. 6 ). Each of the plurality offirst data slices may include a plurality of portions, such that eachportion includes compressed blocks corresponding to a respective one ofthe plurality of channels (e.g., the data slice 650 includes compresseddata 652 of A-channel blocks, compressed data 654 of R-channel blocks,compressed data 656 of G-channel blocks, and compressed data 658 ofB-channel blocks in FIG. 6 ).

In some implementations, the method may further include generating aplurality of second compressed blocks (e.g., compressed blocks 612, 622,632, 642 in FIG. 6 ) by compressing a plurality of second blocks. Eachblock of the plurality of second blocks may correspond to one of theplurality of channels (e.g., compressed R-channel block 612, compressedG-channel block 622, compressed B-channel block 632, and compressedA-channel block 642). The method may further include dividing theplurality of second compressed blocks into a plurality of second dataslices (e.g., data slice 670 in FIG. 6 ). Each of the plurality ofsecond data slices may include a plurality of portions, such that eachportion includes compressed blocks corresponding to a respective one ofthe plurality of channels (e.g., the data slice 670 includes compresseddata 672 of A-channel blocks, compressed data 674 of R-channel blocks,compressed data 676 of G-channel blocks, and compressed data 678 ofB-channel blocks in FIG. 6 ). The plurality of first blocks may have afirst MIP level (e.g., MIP level 0 of the compressed blocks 610, 620,630, 640 in FIG. 6 ) different from a second MIP level of the pluralityof second blocks (e.g., MIP level 1 of the compressed blocks 612, 622,632, 642 in FIG. 6 ).

In some implementations, the method may further generating, based on theplurality of first data slices (e.g., the data slice 1020 in FIG. 10 ),an indirection table (e.g., the indirection table 1040 in FIG. 10 )indicating address information (e.g., offset and length fields in theindirection table 1040 in FIG. 10 ) of compressed blocks in theplurality of first data slices.

Having now described some illustrative implementations, it is apparentthat the foregoing is illustrative and not limiting, having beenpresented by way of example. In particular, although many of theexamples presented herein involve specific combinations of method actsor system elements, those acts and those elements can be combined inother ways to accomplish the same objectives. Acts, elements andfeatures discussed in connection with one implementation are notintended to be excluded from a similar role in other implementations.

The hardware and data processing components used to implement thevarious processes, operations, illustrative logics, logical blocks,modules and circuits described in connection with the embodimentsdisclosed herein may be implemented or performed with a general purposesingle- or multi-chip processor, a digital signal processor (DSP), anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA), or other programmable logic device, discrete gate ortransistor logic, discrete hardware components, or any combinationthereof designed to perform the functions described herein. A generalpurpose processor may be a microprocessor, or, any conventionalprocessor, controller, microcontroller, or state machine. A processoralso may be implemented as a combination of computing devices, such as acombination of a DSP and a microprocessor, a plurality ofmicroprocessors, one or more microprocessors in conjunction with a DSPcore, or any other such configuration. In some implementations,particular processes and methods may be performed by circuitry that isspecific to a given function. The memory (e.g., memory, memory unit,storage device, etc.) may include one or more devices (e.g., RAM, ROM,Flash memory, hard disk storage, etc.) for storing data and/or computercode for completing or facilitating the various processes, layers andmodules described in the present disclosure. The memory may be orinclude volatile memory or non-volatile memory, and may include databasecomponents, object code components, script components, or any other typeof information structure for supporting the various activities andinformation structures described in the present disclosure. According toan exemplary embodiment, the memory is communicably connected to theprocessor via a processing circuit and includes computer code forexecuting (e.g., by the processing circuit and/or the processor) the oneor more processes described herein.

The present disclosure contemplates methods, systems and programproducts on any machine-readable media for accomplishing variousoperations. The embodiments of the present disclosure may be implementedusing existing computer processors, or by a special purpose computerprocessor for an appropriate system, incorporated for this or anotherpurpose, or by a hardwired system. Embodiments within the scope of thepresent disclosure include program products comprising machine-readablemedia for carrying or having machine-executable instructions or datastructures stored thereon. Such machine-readable media can be anyavailable media that can be accessed by a general purpose or specialpurpose computer or other machine with a processor. By way of example,such machine-readable media can comprise RAM, ROM, EPROM, EEPROM, orother optical disk storage, magnetic disk storage or other magneticstorage devices, or any other medium which can be used to carry or storedesired program code in the form of machine-executable instructions ordata structures and which can be accessed by a general purpose orspecial purpose computer or other machine with a processor. Combinationsof the above are also included within the scope of machine-readablemedia. Machine-executable instructions include, for example,instructions and data which cause a general purpose computer, specialpurpose computer, or special purpose processing machines to perform acertain function or group of functions.

The phraseology and terminology used herein is for the purpose ofdescription and should not be regarded as limiting. The use of“including” “comprising” “having” “containing” “involving”“characterized by” “characterized in that” and variations thereofherein, is meant to encompass the items listed thereafter, equivalentsthereof, and additional items, as well as alternate implementationsconsisting of the items listed thereafter exclusively. In oneimplementation, the systems and methods described herein consist of one,each combination of more than one, or all of the described elements,acts, or components.

Any references to implementations or elements or acts of the systems andmethods herein referred to in the singular can also embraceimplementations including a plurality of these elements, and anyreferences in plural to any implementation or element or act herein canalso embrace implementations including only a single element. Referencesin the singular or plural form are not intended to limit the presentlydisclosed systems or methods, their components, acts, or elements tosingle or plural configurations. References to any act or element beingbased on any information, act or element can include implementationswhere the act or element is based at least in part on any information,act, or element.

Any implementation disclosed herein can be combined with any otherimplementation or embodiment, and references to “an implementation,”“some implementations,” “one implementation” or the like are notnecessarily mutually exclusive and are intended to indicate that aparticular feature, structure, or characteristic described in connectionwith the implementation can be included in at least one implementationor embodiment. Such terms as used herein are not necessarily allreferring to the same implementation. Any implementation can be combinedwith any other implementation, inclusively or exclusively, in any mannerconsistent with the aspects and implementations disclosed herein.

Where technical features in the drawings, detailed description or anyclaim are followed by reference signs, the reference signs have beenincluded to increase the intelligibility of the drawings, detaileddescription, and claims. Accordingly, neither the reference signs northeir absence have any limiting effect on the scope of any claimelements.

Systems and methods described herein may be embodied in other specificforms without departing from the characteristics thereof. References to“approximately,” “about” “substantially” or other terms of degreeinclude variations of +/−10% from the given measurement, unit, or rangeunless explicitly indicated otherwise. Coupled elements can beelectrically, mechanically, or physically coupled with one anotherdirectly or with intervening elements. Scope of the systems and methodsdescribed herein is thus indicated by the appended claims, rather thanthe foregoing description, and changes that come within the meaning andrange of equivalency of the claims are embraced therein.

The term “coupled” and variations thereof includes the joining of twomembers directly or indirectly to one another. Such joining may bestationary (e.g., permanent or fixed) or moveable (e.g., removable orreleasable). Such joining may be achieved with the two members coupleddirectly with or to each other, with the two members coupled with eachother using a separate intervening member and any additionalintermediate members coupled with one another, or with the two memberscoupled with each other using an intervening member that is integrallyformed as a single unitary body with one of the two members. If“coupled” or variations thereof are modified by an additional term(e.g., directly coupled), the generic definition of “coupled” providedabove is modified by the plain language meaning of the additional term(e.g., “directly coupled” means the joining of two members without anyseparate intervening member), resulting in a narrower definition thanthe generic definition of “coupled” provided above. Such coupling may bemechanical, electrical, or fluidic.

References to “or” can be construed as inclusive so that any termsdescribed using “or” can indicate any of a single, more than one, andall of the described terms. A reference to “at least one of ‘A’ and ‘B’”can include only ‘A’, only ‘B’, as well as both ‘A’ and ‘B’. Suchreferences used in conjunction with “comprising” or other openterminology can include additional items.

Modifications of described elements and acts such as variations insizes, dimensions, structures, shapes and proportions of the variouselements, values of parameters, mounting arrangements, use of materials,colors, orientations can occur without materially departing from theteachings and advantages of the subject matter disclosed herein. Forexample, elements shown as integrally formed can be constructed ofmultiple parts or elements, the position of elements can be reversed orotherwise varied, and the nature or number of discrete elements orpositions can be altered or varied. Other substitutions, modifications,changes and omissions can also be made in the design, operatingconditions and arrangement of the disclosed elements and operationswithout departing from the scope of the present disclosure.

References herein to the positions of elements (e.g., “top,” “bottom,”“above,” “below”) are merely used to describe the orientation of variouselements in the FIGURES. The orientation of various elements may differaccording to other exemplary embodiments, and that such variations areintended to be encompassed by the present disclosure.

1-18. (canceled)
 19. A method comprising: determining, by one or moreprocessors, a range of values of pixel samples in one or more sub-blocksof a block of image data based at least on a combination of compressionrate and distortion efficiency; identifying, by the one or moreprocessors as one or more data characteristics of the one or moresub-blocks, the range of values, a number of bits per pixel sample, andan offset of a starting endpoint; and compressing, by the one or moreprocessors, a sub-block of the one or more sub-blocks using acompression technique that is identified using the one or more datacharacteristics.
 20. The method according to claim 19, wherein thecompression technique is one of lossless compression, lossy compression,prediction-based compression, or no compression.
 21. The methodaccording to claim 19, wherein the block of image data corresponds toone of a plurality of channels, each channel being one of a colorchannel or an alpha channel.
 22. The method according to claim 21,further comprising: generating a plurality of first compressed blocks bycompressing a plurality of first blocks, each block of the plurality offirst blocks corresponding to one of the plurality of channels; anddividing the plurality of first compressed blocks into a plurality offirst data slices, each of the plurality of first data slices includinga plurality of portions, each portion including compressed blockscorresponding to a respective one of the plurality of channels.
 23. Themethod according to claim 22, further comprising: generating a pluralityof second compressed blocks by compressing a plurality of second blocks,each block of the plurality of second blocks corresponding to one of theplurality of channels; and dividing the plurality of second compressedblocks into a plurality of second data slices, each of the plurality ofsecond data slices including a plurality of portions, each portionincluding compressed blocks corresponding to a respective one of theplurality of channels, wherein the plurality of first blocks have afirst multum in parvo (MIP) level different from a second MIP level ofthe plurality of second blocks.
 24. The method according to claim 22,further comprising: generating, based on the plurality of first dataslices, an indirection table indicating address information ofcompressed blocks in the plurality of data slices.
 25. The methodaccording to claim 19, wherein the range of values is determined suchthat a cost function of the compression rate and the distortionefficiency is minimized.
 26. The method according to claim 19, whereinthe offset is a non-zero value.
 27. A device, comprising: one or moreprocessors, coupled to memory and configured to: determine a range ofvalues of pixel samples in one or more sub-blocks of a block of imagedata based at least on a combination of compression rate and distortionefficiency; identify, as one or more data characteristics of the one ormore sub-blocks, the range of values, a number of bits per pixel sample,and an offset of a starting endpoint; and compress a sub-block of theone or more sub-blocks using a compression technique that is identifiedusing the one or more data characteristics.
 28. The device of claim 27,wherein the compression technique is one of lossless compression, lossycompression, prediction-based compression, or no compression.
 29. Thedevice of claim 27, wherein the block of image data corresponds to oneof a plurality of channels, each channel being one of a color channel oran alpha channel.
 30. The device of claim 29, wherein the one or moreprocessors are further configured to: generate a plurality of firstcompressed blocks by compressing a plurality of first blocks, each blockof the plurality of first blocks corresponding to one of the pluralityof channels; and divide the plurality of first compressed blocks into aplurality of first data slices, each of the plurality of first dataslices including a plurality of portions, each portion includingcompressed blocks corresponding to a respective one of the plurality ofchannels.
 31. The device of claim 30, wherein the one or more processorsare further configured to: generate a plurality of second compressedblocks by compressing a plurality of second blocks, each block of theplurality of second blocks corresponding to one of the plurality ofchannels; and divide the plurality of second compressed blocks into aplurality of second data slices, each of the plurality of second dataslices including a plurality of portions, each portion includingcompressed blocks corresponding to a respective one of the plurality ofchannels, wherein the plurality of first blocks have a first multum inparvo (MIP) level different from a second MIP level of the plurality ofsecond blocks.
 32. The device of claim 30, wherein the one or moreprocessors are further configured to: generate, based on the pluralityof first data slices, an indirection table indicating addressinformation of compressed blocks in the plurality of data slices. 33.The device of claim 27, wherein the range of values is determined suchthat a cost function of the compression rate and the distortionefficiency is minimized.
 34. The device of claim 27, wherein the offsetis a non-zero value.
 35. A non-transitory computer readable mediumstoring program instructions for causing one or more processors to:determine a range of values of pixel samples in one or more sub-blocksof a block of image data based at least on a combination of compressionrate and distortion efficiency; identify, as one or more datacharacteristics of the one or more sub-blocks, the range of values, anumber of bits per pixel sample, and an offset of a starting endpoint;and compress a sub-block of the one or more sub-blocks using acompression technique that is identified using the one or more datacharacteristics.
 36. The non-transitory computer readable medium ofclaim 35, wherein compression technique is one of lossless compression,lossy compression, prediction-based compression, or no compression. 37.The non-transitory computer readable medium of claim 35, wherein theblock of image data corresponds to one of a plurality of channels, eachchannel being one of a color channel or an alpha channel.
 38. Thenon-transitory computer readable medium of claim 35, wherein the rangeof values is determined such that a cost function of the compressionrate and the distortion efficiency is minimized.